Staring Imaging systems have focal plane sensors operating in different spectral bands (e.g., Ultraviolet (UV), Visible, Short Wavelength Infrared (SWIR), Mid Wavelength Infrared (MWIR), or Long Wavelength Infrared (LWIR)) that have difficulty imaging scenes having large dynamic range where image intensity can vary dramatically across the scene. Object space, or the image scene, can be imaged with focal plane sensors which contain many sensor pixels, where each sensor pixel consists of a photodetector with readout provisions. Focal plane sensors can image a scene by virtually dividing it into small areas where each small area is the footprint of a sensor pixel in object space (or the scene). Large dynamic range scenes contain regions which when imaged with sensor pixels can exhibit very large, or average, or very low photosignals. Characteristically, imaging such scenes has three problems: (i) a sensor pixel's overexposure, or underexposure, (ii) digitization of large dynamic range signals (>15 bits), and (iii) poor sensitivity in overexposed or underexposed sensor pixels. Adjusting globally the integration time to optimize the image according the average scene brightness is not an adequate solution for imaging large dynamic range scenes. The average brightness approach dates from film imaging, and is most effective for low dynamic range scenes, but inadequate for imaging large dynamic range scenes.
Recently, another approach has been introduced for imaging high dynamic range scenes which combines multiple images, each taken at a different exposure. Multiple images are merged with a software program into a single combined image. The combined image merges dim image regions (acquired with the longest exposure) with average brightness image regions (acquired with intermediate exposure) with very bright image regions (acquired with short exposure). Software is used to select sensor pixels with the best exposure (signal to noise ratio) and after proper scaling, the software combines the selected sensor pixels into a single surreal image. The combined image produced from multiple exposures and post processing can be effective for imaging large dynamic range scenes, however, it has serious drawbacks. First, multiple images require more time and are appropriate for slow scenes (e.g., where scene images change slowly on the sensor from frame-to-frame), and not faster scenes. Second, combining multiple images requires sensor pixel to sensor pixel registering in multiple images, otherwise blurring can occur. These additional requirements limit the utility of the multiple image approach to situations where: (1) a tripod is used for adequate stability between the camera and scene, and (2) the scene does not change rapidly.
The problem of imaging large dynamic range scenes is illustrated by the example in TABLE 1. The scene's dynamic range entered in the second column is divided into five subranges. Such division illustrates several characteristics of imaging with focal plane sensors containing quantum photodetectors with readout provisions. First, signals from large dynamic range scenes have photosignals which vary over a wide dynamic range (see column 2 in TABLE 1) and the signal to noise ratio varies according to Poisson statistics as the square root of the signal (see column 3 in TABLE 1). Poor sensitivity occurs because the S/N decreases monotonically as the square root of the signal. At the highest signal levels, sensor pixel saturation can occur and this can lead to poor sensitivity. Second, sensitivity dependence of a sensor pixel's photosignal complicates digitizing signals from large dynamic range scenes. Typically, an analog-to-digital (A/D) convertor's least significant bit (LSB) is adjusted to equal approximately the signal's noise level. It is difficult to define a global LSB value for an imaging focal plane sensor because each sensor pixel's noise varies with the photosignal (see column 3 in TABLE 1). This would require varying the A/D LSB for each range (see column 5 in TABLE 1) which raises many complications.
Conventionally, the A/D converters LSB is set at the minimum noise level and that causes inefficient A/D converter operation since significant time is consumed digitizing noise. Third, in large dynamic range images, the signal-to-noise (S/N) ratio is maximum in scene regions with high photosignals and minimum in regions with low photosignals (see column 4 in TABLE 1). This effect translates into noticeable variation in image quality where the best (poorest) image quality is in regions where the sensor pixels have high (low) level photosignal.
TABLE 1 below is an example of the signal levels expected in a focal plane sensors with quantum photodetectors. Each sensor pixel is subjected to the same integration time and field of view. After one integration time, the integrated charge photosignal in each sensor pixel is assumed to vary between 12 and 12,500 photoelectrons. The signal's dynamic range has been divided into five subranges to illustrate how a sensor pixel's noise and S/N ratio varies with signal (see, respectively, third and fourth columns). Digitizing signals with different noise levels complicates selecting an optimal value for the A/D converter's LSB value.
TABLE 1SignalUnscaledUnscaledInstantaneousUnscaled A/DRangeSignal RangeNoise RangeS/N (# Bits)Converter LSB#(electrons)(electrons)Before Scaling(electrons)13,125-12,50056-112<8302 782-3,12528-56 <6153196-782 14-28 <57449-1966-14<43512-49 3-7 <31.5